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Abstract  -  Tracking  maneuvering  targets  is  an 
important  problem.  A  study  was  previously  performed  to 
compare  the  state  estimation  accuracy  of  a  Kalman  filter 
to  an  interacting  multiple  model  (IMM)  for  a 
maneuvering  target.  The  authors  defined  a 
maneuvering  index  to  quantify  the  degree  of 
maneuvering.  Their  study  then  compared  the  state 
estimates  of  the  two  filters  as  a  function  of  this  index. 
Their  results  showed  that  an  IMM  provides  significant 
improvement  over  a  Kalman  filter.  That  study  was 
revisited  and  this  paper  discusses  the  differing  results 
observed.  Our  results  show  that  the  IMM  does  improve 
overall  state  estimations  but  much  less  than  in  the 
previous  study.  This  improvement  is  due  to  the  smaller 
state  estimation  errors  that  the  IMM  provides  over  the 
Kalman  filter  during  the  non-maneuvering  intervals, 
rather  than  the  complete  domination  in  performance  of 
the  IMM  that  the  previous  study  revealed.  As  a  result, 
the  "0.5  rule"  that  the  previous  authors  identified, 
should  be  revised. 
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1  Introduction 

Tracking  maneuvering  targets  is  an  important  problem 
that  occurs  in  many  domains.  In  2001,  Kirubarajan  and 
Bar-Shalom  (K&B)  demonstrated  the  relative  state 
estimation  performance  of  a  Kalman  filter  to  an 
interacting  multiple  model  (IMM)  as  the  target  becomes 
increasingly  more  maneuverable.[l][2]  They  introduced  a 
maneuver  index  metric  which  quantifies  the  degree  of 
maneuverability  of  a  target.  This  index  is  discussed  later. 
They  compared  the  state  estimation  errors  of  these  two 
filters  and  plotted  the  results  as  a  function  of  the 
maneuvering  index.  K&B  showed  that  the  state  estimation 
performance  of  an  IMM  was  considerably  better  (i.e.,  has 
lower  RMS  error)  than  a  Kalman  filter  as  the  target 
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became  more  maneuverable.  This  analysis  led  to  their 
finding  of  the  "0.5  rule"  which  states  that  the  IMM 
performs  better  than  Kalman  once  the  maneuver  index 
exceeded  0.5.[1][2]  K&B  also  plotted  the  estimation 
errors  during  the  intervals  that  the  target  was  maneuvering 
and  during  the  intervals  that  the  target  was  not 
maneuvering.  Their  results  showed  that  the  IMM  produces 
smaller  estimation  errors  than  the  Kalman  filter  regardless 
of  whether  the  target  is  maneuvering  or  not.  This  result 
seemed  unexpected.  While  one  would  expect  their  IMM 
to  provide  smaller  estimation  errors  during  the  non¬ 
maneuvering  intervals,  it  was  not  clear  why  their  IMM 
would  perform  better  than  the  Kalman  filter  during  the 
maneuvering  intervals. 

That  K&B  study  was  re-visited  in  this  study  and  the 
results  are  discussed  in  this  paper.  There  was  one 
difference,  however,  in  this  study.  Instead  of  comparing 
the  Kalman  filter  to  an  IMM,  we  will  introduce  a  Peifect 
IMM  filter  and  use  this  filter  as  the  comparison  to  the 
Kalman  filter.  Using  a  Perfect  IMM  instead  of  an  IMM 
defines  the  lower  bound  on  estimation  errors  possible 
from  an  IMM.  This  paper  will  discuss  the  effort  to  re¬ 
create  the  results  from  the  original  study.  We  will  show, 
unfortunately,  that  K&B’s  results  could  not  be  duplicated. 
We  will  provide  justification  of  why  we  believe  the  results 
we  obtained  are  more  reasonable.  Our  results  will  suggest 
that  the  0.5  rule  that  K&B  identified  should  be  revised. 

2  Introducing  the  Perfect  IMM 

An  IMM  filter  is  effectively  a  Kalman  filter  with  multiple 
motion  (i.e.,  prediction)  models.  At  each  update,  the  IMM 
runs  multiple  filters,  one  for  each  motion  model  and  then 
combines  the  resulting  state  estimations  into  an  overall 
state  estimation  as  a  weighting  of  the  results  from  each 
filter.  The  weights  are  generally  arbitrary  but  based  on 
some  knowledge  about  the  behavior  of  the  target. 
Although  K&B  define  the  weights  used  for  the  IMM  in 
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their  study,  we  will  use  a  Perfect  IMM  (PIMM)  filter 
instead.  A  PIMM  filter  is  a  theoretical  IMM-like  filter  that 
cannot  be  realized  for  any  real  system,  but  is  useful  for  a 
simulated  system.  In  an  IMM,  the  goal  is  to  apply  the 
right  motion  model  at  the  right  time  (i.e.,  update).  In 
simulation,  the  true  motion  of  the  target  is  known  (to  the 
simulation),  so  the  PIMM  is  told  which  motion  model  to 
use  for  each  update.  Thus,  the  selection  of  the  correct 
motion  model  is  always  performed  perfectly  and  hence  its 
name,  Perfect  IMM.  Since  a  Perfect  IMM  always  selects 
the  right  motion  model,  it  defines  the  lower  bound  on  the 
estimation  errors  of  any  corresponding  IMM  making  it 
useful  for  comparison  studies  of  IMM  performance. 
Furthermore,  since  the  Kalman  filter  effectively  defines 
the  upper  bound  on  the  estimation  errors  of  any  IMM,  we 
know  that  any  real  IMM  will  perform  somewhere  between 
these  two  bounds. 

For  this  study,  a  two-model  PIMM  is  used  that  is  similar 
to  the  two-model  IMM  that  K&B  used  in  their  study.  The 
two  models  are  identical  except  for  process  noise.  K&B 
defined  a  low  process  noise  for  when  the  target  is  deemed 
not  maneuvering  and  a  high  process  noise  for  when  the 
target  is  deemed  maneuvering.  The  same  process  noise 
models  were  used  in  the  PIMM.  The  parameters  for  the 
process  noise  will  be  given  later. 

3  The  Target  Motion  Model 

In  the  K&B  study,  a  simplistic  two-dimensional  target 
model  is  used  where  the  acceleration  error  distributions  in 
the  x  and  y  directions  were  identical.  They  assumed  on 
each  update,  the  target  randomly  selects  an  acceleration 
from  a  distribution  with  a  large  or  small  variance, 
depending  if  the  target  is  maneuvering  or  maintaining 
(nearly)  constant  motion  during  this  time  update.  K&B 
also  assumed  that  the  target  maneuvers  were  synchronized 
with  the  measurement  updates.  Thus,  the  target  always 
starts  a  maneuver  at  the  measurement  update  time  and 
continues  with  that  maneuver  until  the  next  update  time. 
Note  that  aligning  the  maneuver  to  the  update  time  is 
unrealistic  since  these  events  are  independent  of  each 
other.  Although  this  is  unrealistic,  for  consistency,  we  use 
the  same  approach  in  this  study.  It  is  suspected,  however, 
that  making  the  maneuver  times  independent  of  the 
measurement  update  times  would  not  have  any  large 
affect  on  the  results  of  the  study. 

Based  on  these  assumptions,  a  simple  motion  model  can 
be  derived  as  follows.  Let  T  be  the  time  between  updates. 
Let  ax  and  ay  be  the  randomly  selected  x  and  y 
accelerations  respectively,  for  the  ith  update.  Then,  the 
target  state  for  the  i'h  update  would  be: 


Xj  =  xi_1  +  axT 

(1) 

Vi  =  Vt- 1  +  ctyT 

(2) 

xt  =  xi_1  +  xi_1  +  ^axT2 

(3) 

Vi  =  Vi-i  +  yi- 1  +  \a-yT2 

(4) 

K&B  state  that  the  accelerations  are  selected  from  a  zero- 
mean  distribution  with  standard  deviation,  aa.  The  type  of 
distribution  was  not  stated  but  assumed  to  be  Gaussian. 


4  Comparison  Test  Method 

Test  scenarios  will  be  constructed  identically  to  the  ones 
used  in  the  K&B  study.  The  initial  position  of  the  target 
will  be  at  (25,000  m,  10,000  m)  with  an  initial  velocity  of 
(-120  m/s,  0  m/s).  Each  scenario  lasts  seven  minutes  of 
simulated  time.  During  the  first,  third,  fifth,  and  seventh 
minute,  the  target  moves  with  nearly  constant  velocity 
with  process  noise  standard  deviation  ua  =  0.2  m/s2. 
During  the  second,  fourth,  and  sixth  minute  the  target 
maneuvers  with  a  (fixed)  process  noise  level  that  is  varied 
from  c ra  =  0.4  m/s2  to  10.0  m/s2,  in  increments  of  0.4  m/s2. 
The  comparison  graphs  however,  plot  the  state  estimation 
root-mean- squared  (RMS)  errors  as  a  function  of  the 
maneuver  index,  X,  just  as  was  done  in  the  K&B  study. 
The  maneuver  index  is  a  unit-less  quantity  formed  as  the 
ratio  of  the  uncertainty  in  the  target  motion  over  the 
uncertainty  in  the  measurement.  The  index  is  defined  as: 


°w 


where  aa  is  the  process  noise  standard  deviation  (in  each 
dimension),  T  is  the  time  between  updates,  and  aw  is  the 
measurement  error  standard  deviation  (in  each 
dimension). 

Like  the  K&B  study,  the  measurement  errors  in  the  x  and 
y  dimension  are  assumed  to  both  be  normally  distributed 
with  erw  =  100  m  in  each  dimension.  The  time  between 
updates  is  T  =  5  sec. 

The  results  of  the  comparison  will  be  summarized  as  a  set 
of  plots  describing  the  position  and  velocity  estimation 
RMS  errors,  using  Monte  Carlo  analysis.  The  original 
study  used  100  Monte  Carlo  iterations  for  each  process 
noise  standard  deviation.  To  make  the  plots  smoother,  this 
study  uses  1000  Monte  Carlo  iterations.  The  average  RMS 
error,  Eavg  is: 


E 


avg 


(6) 


where  m  =  number  of  Monte  Carlo  iterations  =  1000,  n  = 
number  of  measurement  updates  per  scenario,  and 

efj  =  ( xtj  —  xtj)  +  (yy  —  yy)  is  the  computed  error  of 
the  ith  measurement  update  on  the  jth  Monte  Carlo 
iteration. 


4.1  Preparing  the  Kalman  Filter 

The  Kalman  filter  for  this  study  follows  identically  with 
the  one  in  the  K&B  study.  It  is  assumed  that  the  true  state 
of  the  target  at  time  tk  is 

x(tfc)  =  [x(tk)  x(tk)  y(tk )  y(tk)]T  (7) 

where  x(tk),y(tk)  is  the  position  and  x(tk),y(tk)  is  the 
velocity  in  the  x,y  coordinates,  respectively.  Like  the 
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K&B  study,  the  covariance  matrix  R  of  measurement 
noise  is: 


R  = 


(8) 


The  plant  noise  matrix  Q  is  also  identical  to  the  one  used 
in  the  K&B  study. 
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where  era  is  set  to  0.8  of  the  process  noise  standard 
deviation  used  during  the  maneuvering  intervals. 


4.2  Preparing  the  PIMM  Filter 

In  the  K&B  study,  the  IMM  simply  employs  two  different 
plant  noise  matrices;  one  for  when  the  target  is  deemed  to 
be  maneuvering  and  the  other  when  the  target  is  deemed 
to  not  be  maneuvering.  The  plant  noise  matrix  for  the  non¬ 
maneuvering  target  is  found  using  the  low  process  noise 
aa  =  0.2  m/s2.  The  plant  noise  matrix  for  the  maneuvering 
target  is  found  using  the  process  noise  selected  during  the 
maneuvering  interval.  In  this  study,  the  PIMM  filter 
follows  similarly  using  the  two  different  plant  noise 
matrices,  except  it  is  told  which  plant  noise  matrix  to  use 
at  each  update. 

5  Results 


Figures  1  through  4  are  the  results  obtained  from  this 
study.  The  figures  directly  correspond  to  the  ones  in  the 
K&B  study.[l]  Figs  la  and  lb  plot  the  peak  position 
estimation  RMS  errors  and  peak  velocity  estimation  RMS 
errors  respectively,  over  the  entire  seven-minute  scenario 
as  a  function  of  the  maneuver  index.  The  blue  dotted  line 
plots  the  RMS  errors  from  the  Kalman  filter.  The  red  solid 
line  plots  the  errors  from  the  PIMM.  Like  the  K&B  study, 
we  show  that  the  peak  estimation  errors  from  the  Kalman 
filter  are  larger  than  those  from  the  PIMM  filter. 
Flowever,  our  results  show  the  differences  between  these 
two  filters  are  much  less  than  what  K&B  obtained. 

Fig.  la  shows  that  the  peak  position  errors  for  the  Kalman 
at  L=  2.5  is  about  155  m.  K&B  show  this  peak  error  to  be 
around  300  m,  about  twice  as  large.  Furthermore,  the 
figure  shows  the  PIMM  error  is  about  145  m,  which  is 
only  ~6%  improvement.  The  K&B  study  shows  the  IMM 
error  to  be  about  180  m,  about  a  40%  improvement.  The 
peak  errors  for  the  velocity  estimation  are  also  different 
from  K&B’s  result.  Fig.  lb  shows  the  peak  velocity  errors 
to  be  approximately  the  same  between  the  two  filters  with 
a  difference  growing  to  only  ~5  m/s.  K&B  show  the 
Kalman  estimation  error  to  be  nearly  three  times  larger 
than  the  IMM  error.  Additionally,  K&B  show  that  the 
velocity  estimation  errors  for  the  IMM  grow  to  only  ~12 
m/s,  which  is  about  four  times  smaller  than  the  errors  we 
obtained. 


Figs  2a  and  2b  plot  the  overall  position  estimation  errors 
and  overall  velocity  estimation  errors  respectively,  over 
the  entire  seven-minute  scenario  as  a  function  of  the 
maneuver  index.  Fig.  2a  shows  the  position  estimation 
errors  for  the  Kalman  filter  leveling  off  to  ~1 30  m.  The 
K&B  results  show  the  position  estimation  error  climbing 
to  210  m  and  seemingly  continuing  to  climb.  Their 
Kalman  filter  position  errors  are  much  larger  than  the 
errors  in  the  measurements.  Their  results  seem  suspect 
since  this  would  mean  that  using  the  measurements 
themselves  would  produce  better  position  estimates  than 
using  the  Kalman  filter  estimates.  There  is  close 
agreement  (115  mvs  120  m)  for  the  position  estimates  for 
the  IMM,  except  our  results  show  a  clear  leveling  off  of 
error  while  the  K&B  results  show  an  upward  trend.  As  a 
result,  their  IMM  position  errors  seem  suspect  as  well. 


Maneuver  Index 


Figure  1.  Base  case.  Peak  RMS  errors  of  Kalman  versus  PIMM, 
(a)  Position  errors;  (b)  Velocity  errors. 


Like  the  peak  velocity  estimation  error  plot,  Fig  2b  shows 
minimal  difference  between  the  two  filters;  again  rising  to 
a  difference  of  only  ~5  m/s.  The  K&B  study,  on  the  other 
hand,  shows  the  velocity  estimation  error  from  the 
Kalman  rising  to  twice  the  error  of  the  IMM.  Flowever, 
our  velocity  errors  from  the  Kalman  filter  are  about  twice 
as  large  as  those  from  the  K&B  study,  and  our  velocity 
errors  from  the  IMM  are  about  three  times  larger. 

Figs  3a  and  3b  plot  the  position  estimation  errors  and 
velocity  estimation  errors  respectively,  during  the  non¬ 
maneuvering  (a.k.a.,  uniform  motion)  intervals  of  the 
scenario,  as  a  function  of  the  maneuver  index.  It  is  during 
these  intervals  where  the  PIMM  is  expected  to  outperform 
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Kalman  since  the  PIMM  can  use  the  smaller  process 
noise.  As  expected,  the  plots  show  the  dramatic 
differences  in  the  errors  between  the  Kalman  and  the 
PIMM.  The  results  in  Fig  3a  are  similar  to  those  in  the 
K&B  except  they  show  their  two  filters  each  rise  to  about 
15  m  less.  The  velocity  estimation  errors  shown  in  Fig.  3b 
are  different  than  K&B’s  result,  but  there  is  agreement 
that  the  1MM  estimation  errors  are  dramatically  improved 
over  the  Kalman.  It  is  suspected  that  the  y-axis  from  the 
K&B  study  is  mislabeled  as  it  shows  the  Kalman  filter 
velocity  errors  grow  to  only  ~4  m/s. 


Maneuver  Index 


Maneuver  Index 


Figure  2.  Base  case.  Overall  RMS  errors  of  Kalman  versus 
PIMM,  (a)  Position  errors;  (b)  Velocity  errors. 

Figs  4a  and  4b  plot  the  position  estimation  errors  and 
velocity  estimation  errors  respectively,  during  the 
maneuvering  intervals  of  the  scenario  as  a  function  of  the 
maneuver  index.  Note  that  there  is  little  difference 
between  the  two  filters.  This  is  what  should  be  expected 
since  the  two  filters  are  virtually  identical  during  these 
maneuvering  time  intervals.  The  K&B  study,  on  the  other 
hand,  shows  a  dramatic  improvement  of  the  1MM  over  the 
Kalman  as  the  maneuvering  index  increases.  In  fact,  they 
show  the  position  errors  for  both  filters  grow  larger  than 
the  errors  in  the  measurements  themselves.  Their  position 
error  for  the  IMM  grew  to  — 1 80  m  and  the  position  error 
for  the  Kalman  filter  grew  to  ~260  m.  Here  again  their 
results  suggest  it  would  be  (much)  better  to  ignore  the 
filter  and  just  use  the  measurements.  The  fact  that  they 
showed  the  IMM  offered  dramatic  improvement  over  the 
Kalman  filter  is  questionable.  Since  the  filters  are  nearly 


identical  during  these  intervals,  it  is  unclear  how  they 
could  obtain  large  differences  between  the  two  filters. 


Maneuver  Index 


Dotted  Blue=Kalman,  Solid  Red=PerfectlMM 


Figure  3.  Base  case.  RMS  errors  of  Kalman  versus  PIMM  during 
the  non-maneuvering  intervals,  (a)  Position  errors;  (b)  Velocity 
errors. 
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Figure  4.  Base  case.  RMS  errors  of  Kalman  versus  PIMM  during 
the  maneuvering  intervals,  (a)  Position  errors;  (b)  Velocity 
errors. 
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Note  that  since  the  two  filters  are  resulting  in  nearly 
identical  estimation  errors,  it  means  that  the  Kalman  filter 
is  producing  the  lower  bound  on  the  errors  during  these 
maneuvering  intervals. 

6  Conclusion 

Based  on  our  results,  we  agree  with  the  K&B  study  that 
the  1MM  is  superior  to  the  Kalman  filter,  but  only  because 
an  1MM  can  lower  its  process  noise  during  the  non¬ 
maneuvering  intervals  to  provide  tighter  estimates  during 
those  times.  The  overall  performance  improvement  that 
the  1MM  achieves  is  based  on  exploiting  the  intervals  of 
time  when  the  target  is  not  maneuvering.  Thus,  if  the 
target  is  constantly  maneuvering,  or  at  least,  has  only 
short  periods  when  it  is  not  maneuvering,  then  the  1MM 
will  offer  little  improvement. 

It  is  important  to  recall  that  this  study  introduced  and  used 
a  Perfect  1MM  as  the  comparison.  As  mentioned  earlier, 
this  filter  provides  a  lower  bound  on  the  estimation  errors 
of  any  1MM.  The  study  showed  that  during  the 
maneuvering  intervals,  the  Kalman  filter  state  estimates 
were  as  good  as  the  PIMM,  so  therefore  a  Kalman  filter 
provides  this  lower  bound.  It  was  only  during  the  non¬ 
maneuvering  intervals  where  the  PIMM  had  significantly 
smaller  estimation  errors  than  the  Kalman  filter.  As  a 
result,  we  are  not  inclined  to  agree  with  K&B's  '0.5  rule' 
of  when  it  is  better  to  use  an  1MM  over  a  simpler  Kalman 
filter.  Instead  we  offer  that  the  1MM  will  outperform  the 
Kalman  filter  only  if  a  maneuvering  target  does  not 
maneuver  for  times  long  enough  for  the  1MM  to  recognize 
that  fact. 

The  results  from  this  study  should  help  guide  tracking 
system  design,  especially  when  considering  continually 
maneuvering  targets. 
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